Transition flight control system design for fixed-wing VTOL UAV: a reinforcement learning approach

Tilt-rotor vertical takeoff and landing aerial vehicles have been gaining popularity in urban air mobility applications because of their ability in performing both hover and forward flight regimes. This hybrid concept leads energy efficiency which is quite important to obtain a profitable and sustainable operation. However, inherent dynamical nonlinearities of the aerial platform requires adaptation capability of the control systems. In addition, transition flight phase should be planned carefully not only for a profitable operation but also for a safe transition between flight regimes in the urban airspace. In this paper, transition flight phase of a tilt-rotor vertical-takeoff-and-landing unmanned aerial vehicle (UAV) is studied. Low-level flight control systems are designed based on adaptive dynamic inversion methodology to compensate aerodynamic effects during the transition phase. Reinforcement learning method is utilized to provide safety and energy efficiency during the transition flight phase. An actor-critic agent is utilized and trained by using deep deterministic policy gradient algorithm to augment the collective channel of the UAV. This augmentation on the collective input is used to adjust flight path angle of the UAV which results in adjusting the angle of attack when pitch angle is zero. By using this relationship, it is proposed to generate aerodynamic lift force and perform transition flight with minimum altitude change and energy usage. Simulation results show that the agent reduces the collective signal level as the aerodynamic lift force is created in the descent flight phase. This affects overall system efficiency, reduces operational costs and makes the enterprise more profitable

Reinforcement learning based closed-loop reference model adaptive flight control system design

In this study, we present a reinforcement learning (RL)-based flight control system design method to improve the transient response performance of a closed-loop reference model (CRM) adaptive control system. The methodology, known as RL-CRM, relies on the generation of a dynamic adaption strategy by implementing RL on the variable factor in the feedback path gain matrix of the reference model. An actor-critic RL agent is designed using the performance-driven reward functions and tracking error observations from the environment. In the training phase, a deep deterministic policy gradient algorithm is utilized to learn the time-varying adaptation strategy of the design parameter in the reference model feedback gain matrix. The proposed control structure provides the possibility to learn numerous adaptation strategies across a wide range of flight and vehicle conditions instead of being driven by high-fidelity simulators or flight testing and real flight operations. The performance of the proposed system was evaluated on an identified and verified mathematical model of an agile quadrotor platform. Monte-Carlo simulations and worst case analysis were also performed over a benchmark helicopter example model. In comparison to the classical model reference adaptive control and CRM-adaptive control system designs, the proposed RL-CRM adaptive flight control system design improves the transient response performance on all associated metrics and provides the capability to operate over a wide range of parametric uncertainties

AUTOFLY-Aid: Flight Deck Automation Support with Dynamic 4D Trajectory Management for Responsive and Adaptive Airborne Collision Avoidance

AUTOFLY-Aid Project aims to develop and demonstrate novel automation support algorithms and tools to the flight crew for flight critical collision avoidance using “dynamic 4D trajectory management”. The automation support system is envisioned to improve the primary shortcomings of TCAS, and to aid the pilot through add-on avionics/head-up displays and reality augmentation devices in dynamically evolving collision avoidance scenarios. The main theoretical innovative and novel concepts to be developed by AUTOFLY-Aid project are a) design and development of the mathematical models of the full composite airspace picture from the flight deck’s perspective, as seen/measured/informed by the aircraft flying in SESAR 2020, b) design and development of a dynamic trajectory planning algorithm that can generate at real-time (on the order of seconds) flyable (i.e. dynamically and performance-wise feasible) alternative trajectories across the evolving stochastic composite airspace picture (which includes new conflicts, blunder risks, terrain and weather limitations) and c) development and testing of the Collision Avoidance Automation Support System on a Boeing 737 NG FNPT II Flight Simulator with synthetic vision and reality augmentation while providing the flight crew with quantified and visual understanding of collision risks in terms of time and directions and countermeasures

Flight Deck Automation Support with Dynamic 4D Trajectory Management for ACAS: AUTOFLY-AID

AUTOFLY-Aid Project aims to develop and demonstrate novel automation support algorithms and tools to the flight crew for flight critical collision avoidance using “dynamic 4D trajectory management”. The automation support system is envisioned to improve the primary shortcomings of TCAS, and to aid the pilot through add-on avionics/head-up displays and reality augmentation devices in dynamically evolving collision avoidance scenarios. The main theoretical innovative and novel concepts to be developed by AUTOFLY-Aid Project are a) design and development of the mathematical models of the full composite airspace picture from the flight deck’s perspective, as seen/measured/informed by the aircraft flying in SESAR 2020 b) design and development of a dynamic trajectory planning algorithm that can generate at real-time (on the order of seconds) flyable (i.e. dynamically and performance-wise feasible)alternative trajectories across the evolving stochastic composite airspace picture (which includes new conflicts, blunder risks, terrain and weather limitations) and c) development and testing of the Collision Avoidance Automation Support System on a Boeing 737 NG FNPT II Flight Simulator with synthetic vision and reality augmentation while providing the flight crew with quantified and visual understanding of collision risks in terms of time and directions and countermeasures

Safe motion planning and learning for unmanned aerial systems

To control unmanned aerial systems, we rarely have a perfect system model. Safe and aggressive planning is also challenging for nonlinear and under-actuated systems. Expert pilots, however, demonstrate maneuvers that are deemed at the edge of plane envelope. Inspired by biological systems, in this paper, we introduce a framework that leverages methods in the field of control theory and reinforcement learning to generate feasible, possibly aggressive, trajectories. For the control policies, Dynamic Movement Primitives (DMPs) imitate pilot-induced primitives, and DMPs are combined in parallel to generate trajectories to reach original or different goal points. The stability properties of DMPs and their overall systems are analyzed using contraction theory. For reinforcement learning, Policy Improvement with Path Integrals (PI2) was used for the maneuvers. The results in this paper show that PI2 updated policies are a feasible and parallel combination of different updated primitives transfer the learning in the contraction regions. Our proposed methodology can be used to imitate, reshape, and improve feasible, possibly aggressive, maneuvers. In addition, we can exploit trajectories generated by optimization methods, such as Model Predictive Control (MPC), and a library of maneuvers can be instantly generated. For application, 3-DOF (degrees of freedom) Helicopter and 2D-UAV (unmanned aerial vehicle) models are utilized to demonstrate the main results

A comprehensive flight plan risk assessment and optimization method considering air and ground risk of UAM

Inspired by risk analysis assistance service and flight plan preparation / optimization service in U-space service, this paper investigates a flight plan risk assessment and optimization method for future urban air mobility. The quantitative risk assessment of the flight plan is divided into two parts: the ground and air risks of the flight plan. After evaluating the risk of the flight plan, optimization suggestions are given to guide the path planning algorithm to optimize the flight plan at low risk. The quantitative risk assessment of the flight plan corresponds to risk analysis assistance service in U-space service, and the procedure to give optimization suggestions correspond to flight plan preparation / optimization service in U-space service. This paper selects the task scenario of logistics drone cargo transportation and carries out risk assessment on the specific flight plan. From the assessment results, when the flight plan crosses the pedestrian intensive area on the ground or the road with high-speed vehicles, the risk value of the corresponding flight plan segment increases significantly. When the flight plan segment approaches the area near the airport or intersects with other UAM participants with the same mission time window, the corresponding risk value is also high. After obtaining the risk assessment results, the targeted optimization suggestions are given to guide the path planning algorithm to optimize the flight plan at low risk. The risk of the optimized flight plan has been significantly reduced

Conflict probability based strategic conflict resolution for UAS traffic management

In this paper, we present a strategic conflict resolution method based on the conflict probability estimation, in the context of Unmanned Aircraft System (UAS) Traffic Management. We first elaborate a classic approach for flight trajectory generation in a designated realistic airspace environment, which is then smoothed by B-spline algorithm to achieve higher realism. The trajectories are extended to 4-dimensional Operational Volumes (OV) following the current UTM development visions. This forms the basis for performing a coarse conflict screening process, as the initial part for conflict detection, primarily based on identifying any OVs overlapping in temporal and spatial. Next, we look into the captured OVs and apply a well-studied conflict probability estimation approach, which contributes to a refined and more accurate conflict detection outcome. To resolve the potential conflicts, we propose two models including First-Come, First-Served (FCFS) and optimisation, both embedded with the probability-based conflict detection. In the FCFS approach, flights are delayed in the order of their submission, while the optimisation model aims at cherry-picking flights to seek the optimal solution. Numerical experiments with various case studies are performed to assess the effects with and without such probability concern, as well as different implementation strategies in real world. Results suggest that, allowing OVs’ overlapping to some extent does not necessarily incur conflict over an acceptable probability, whereas the efficiency of airspace use could be improved.This work was partially funded by the SESAR JU under grant agreement No 101017702, as part of the European Union’s Horizon 2020 research and innovation programme: AMU-LED (Air Mobility Urban - Large Experimental Demonstrations)

A risk-based UAM airspace capacity assessment method using Monte Carlo simulation

Inspired by risk analysis assistance service and dynamic capacity management service in U-space service, this paper investigates a risk-based UAM airspace capacity assessment method using Monte Carlo simulation for future urban air mobility. The quantitative risk assessment of the flight plan is divided into three parts: the ground / air risks of the flight plan and the mid-air collision risk between UAM. Using the comprehensive risk assessment method, this paper generates several simulation scenarios in the airspace to be evaluated in terms of the type of participants, the presence of the detect and avoid system, and the total number of participants in the airspace, conducts Monte Carlo simulations, and records the simulation data for analysis. Through the analysis of simulation data, it is found that the maximum risk of UAM in airspace increases with the increase of the number of airspace invaders and the total number of UAM. However, the maximum risk of UAM in airspace decreases when the aircraft in airspace contains the detection and avoid system with the same other conditions. Based on simulation data, this paper informatively proposes the concept of a 3D risk surface and a risk-based airspace capacity envelope, using the horizontal surface formed by a specific risk threshold to cut the 3D risk surface to form an airspace capacity envelope, which visually describes the number of aircraft that can be contained in the airspace under a specific risk threshold

An integrated approach for on-demand dynamic capacity management service in U-space

This paper presents an integrated approach for on-demand Dynamic Capacity Management (DCM) service to be offered in U-space. The approach involves three main threads, including flight planning (demand), airspace configuration (capacity) and demand-capacity balancing (DCB). The flight planning thread produces UAS (unmanned aerial systems) trajectories for each flight that together reflect the estimated traffic demand. The airspace configuration thread defines the fundamental airspace structure and proposes dynamic adjustment schemes that determine the capacity distribution. It also enables the flight planning to reschedule alternative trajectory options to route away from possible congested areas. The last DCB thread takes the previous inputs and then computes for the optimal slot allocation and trajectory selection, as well as the optimal airspace configuration. Simulation case studies have been performed through mimicking an envisioned U-space operating scenario. Results suggest that the integrated approach can achieve the best outcome in almost all the key performance areas than any other cases where only partial functions are realised